Cross Entropy Method for Multiclass Support Vector Machine

In this paper, an importance sampling method – cross entropy method is presented to deal with solving support vector machines (SVM) problem for multiclass classification cases. Using one-against-rest (OAR) and one-against-one (OAO) approaches, several binary svm classifiers are constructed and combined to solve multiclass classification problems. For each binary SVM classifier, the cross entropy method is applied to solve dual Lagrange SVM optimization problem to find the optimal or at least near optimal solution which is Lagrange multipliers, in the feature space through kernel map. For the meantime only RBF kernel function is investigated intensively. Experiments are done on four real world data sets. The results show one-against-rest produces better results than one-against-one in terms of computing time and generalization error. In addition, applying cross entropy method on multiclass SVM produces comparable results to the standard quadratic programming SVM in terms of generalization error.